Automatic adjustment logical positions of multiple screen

ABSTRACT

A method and a computer system are provided for automatically setting the logical positions of multiple screen displays. A computer system may comprise a plurality of display devices, at least one image capturing device, and a controller. The controller may be coupled to the display devices and image capturing devices. The adjustment module may be adapted to adjust the plurality of the display settings.

BACKGROUND OF THE INVENTION

The present invention relates generally to methods and systems foradjusting logical positions of multiple screen displays and, morespecifically, to methods and systems for assessing the position ofobjects that facing displays such as a user's face and affecting thelogical positions of multiple screen displays based on such assessments.

Multiple screen display computer systems, i.e., a computer system withmore than one display or monitor, are becoming more prevalent. Theirprevalence is due, in large part, to the fact that the typical graphicdisplay subsystem provides support for more than one monitor and thecost for a second monitor is relatively small. Another factor is that amultiple screen computer system offers a computer user more area uponwhich information and work may be displayed. With a greater displayarea, the computer user may spend less time cycling through overlappingwindows, frequently referred to as “thrashing,” to find information thatmay lie hidden under the overlapping windows.

Most typical example of multiple screen displays is dual displays usinga primary monitor and a secondary monitor. To arrange dual displays ormonitors, users typically configure one monitor as a primary monitor,which displays the logon dialog box when a computer is started. Asecondary monitor may be connected to a computer system associated withthe primary monitor via a cable or wirelessly. As an example, theprimary monitor is put on the right hand side and a secondary monitor isput on the left hand side in front of a user with a left edge of theprimary monitor abutting the right edge of the secondary monitor. Alogical position of the secondary monitor may be configured as a leftmonitor through a control panel of the computer system, for example. Thelogical position may refer to a computer setting to define relativephysical positions of monitors.

When dual monitors are switched in positions, that is, in the aboveexample, the primary monitor is put on the left hand side and asecondary monitor is put on the right hand side with a right edge of theprimary monitor abutting a left edge of the secondary monitor, thelogical positions of monitors have to be changed in the computer systemaccordingly. If not, users have to move mouse cursor from the primarymonitor to the secondary monitor through the left edge of the primarymonitor screen to the secondary monitor screen. It is not convenient fora user to change the logical positions of monitors in the computersystem each time the monitors switch positions. When there are more thantwo monitors, it is difficult to configure the monitor settings.

Therefore, it can be seen that there is a need to automatically adjustthe logical positions in a computer system when multiple monitors areused.

SUMMARY

In one aspect, a computer system comprises a plurality of displaydevices having at least one main display device and one secondarydisplay device; a plurality of image capturing devices carried by theplurality of display devices; a controller coupled to at least onedisplay device and the image capturing devices; and an adjustment moduleadapted to set logical positions of the plurality of display devices.

In another aspect, a method of setting a multiple screen computer systemmay comprise a plurality of display devices operatively associated withat least one computing device; capturing an object image; analyzing theobject image; and setting logical positions of the plurality of displaydevices based on the analysis of the object image.

In a further aspect, a computer readable medium may have computer usableprogram code comprising computer program code configured to launch asub-display application executing an object recognition (e.g. facerecognition) program; and computer program code configured to setlogical positions of the plurality of display devices according to theobject images received by the object recognition program.

These and other features, aspects and advantages of the presentinvention will become better understood with reference to the followingdrawings, description and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an exemplary embodiment of a computersystem having a multiple-screen display;

FIG. 2A is a front view of an exemplary embodiment of a computer systemhaving a secondary monitor connected to a main monitor by a cable;

FIG. 2B is a flow chart illustrating steps of a method of settingmultiple screen logical positions when multiple screens are connectedvia a cable;

FIG. 3A is a front view of an exemplary embodiment of a computer systemhaving a wireless secondary display; and

FIG. 3B is a flow chart illustrating steps of a method of settingmultiple screen logical positions when multiple screens are connectedwirelessly.

DETAILED DESCRIPTION OF EMBODIMENTS

The following detailed description is of the best currently contemplatedmodes of carrying out exemplary embodiments. The description is not tobe taken in a limiting sense, but is made merely for the purpose ofillustrating the general principles, since the scope of the embodimentsis best defined by the appended claims.

Various inventive features are described below that can each be usedindependently of one another or in combination with other features.

Broadly, exemplary embodiments provide methods and systems for detectingand setting multiple-screen displays. When two or more monitors areconnected to a computer, the computer provides a monitor setting(logical position adjustment) to describe the physical position of themonitors relative to each other. The exemplary embodiments may use animage capturing device that captures a face image of a user in front ofthe multiple-screen displays. Face recognition technology may be used todetect the position of an eye or nose by analyzing brightness. The facerecognition technology may further be used to calculate faceorientation. This may allow for automatic logical position adjustment ofthe multiple screen displays based upon captured face images andcalculated face orientation. Exemplary embodiments optimize theuser-friendliness of communication devices from a screen displayperspective. In exemplary embodiments, multiple-screen displays may beset according to their physical orientation relative to the user.

Exemplary embodiments may include a controller which is coupled to atleast one display device and at least one image capturing device. Thecontroller may comprise a monitoring module and an adjustment module.The monitoring module may capture and analyze an object image, such as auser's face. The adjustment module may be adapted to set the logicalpositions of a plurality of display devices.

Exemplary embodiments may take the form of an entire hardwareembodiment, an entire software embodiment (including firmware, residentsoftware, micro-code, etc.) or an embodiment combining software andhardware aspects that may all generally be referred to herein as a“circuit,” “module” or “system.” Furthermore, exemplary embodiments maytake the form of a computer program product embodied in any tangiblemedium of expression having computer-usable program code embodied in themedium.

Any combination of one or more computer usable or computer readablemedium(s) may be utilized. The computer-usable or computer-readablemedium may be, for example but not limited to, an electronic, magnetic,optical, electromagnetic, infrared, or semiconductor system, apparatus,device, or propagation medium. More specific examples (a non-exhaustivelist) of the computer-readable medium would include the following: anelectrical connection having one or more wires, a portable computerdiskette, a hard disk, a random access memory (RAM), a read-only memory(ROM), an erasable programmable read-only memory (EPROM or Flashmemory), an optical fiber, a portable compact disc read-only memory(CD-ROM), an optical storage device, a transmission media such as thosesupporting the Internet or an intranet, or a magnetic storage device.Note that the computer-usable or computer-readable medium could even bepaper or another suitable medium upon which the program is printed, asthe program can be electronically captured, via, for instance, opticalscanning of the paper or other medium, then compiled, interpreted, orotherwise processed in a suitable manner, if necessary, and then storedin a computer memory. In the context of this document, a computer-usableor computer-readable medium may be any medium that can contain, store,communicate, propagate, or transport the program for use by or inconnection with the instruction performance system, apparatus, ordevice. The computer-usable medium may include a propagated data signalwith the computer-usable program code embodied therewith, either inbaseband or as part of a carrier wave. The computer usable program codemay be transmitted using any appropriate medium, including but notlimited to wireless, wired, wire line, optical fiber cable, radiofrequency (RF), and the like.

Computer program code for carrying out operations of exemplaryembodiments may be written in any combination of one or more programminglanguages, including an object oriented programming language such asJava™ Smalltalk™, C++ or the like, and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer, or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Exemplary embodiments are described below with reference to flowchartillustrations and/or block diagrams of methods, apparatus (systems) andcomputer program products. It will be understood that each block of theflowchart illustrations and/or block diagrams, and combinations ofblocks in the flowchart illustrations and/or block diagrams, can beimplemented by computer program instructions.

These computer program instructions may be provided to a processor of ageneral purpose computer, special purpose computer, or otherprogrammable data processing apparatus to produce a machine, such thatthe instructions, which execute via the processor of the computer orother programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

These computer program instructions may also be stored in acomputer-readable medium that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablemedium produce an article of manufacture including instruction meanswhich implement the function/act specified in the flowchart and/or blockdiagram block or blocks. The computer program instructions may also beloaded onto a computer or other programmable data processing apparatusto cause a series of operational steps to be performed on the computeror other programmable apparatus to produce a computer implementedprocess such that the instructions which execute on the computer orother programmable apparatus provide processes for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

Referring to FIG. 1, a multiple-screen display system 100 may comprise acontroller 105, which includes a monitoring module 110 and an adjustmentmodule 120. The monitoring module 110 may capture and analyze an objectimage, such as a user's face. The adjustment module 120 may be adaptedto set logical positions of a plurality of display devices.

FIG. 2A refers to an exemplary embodiment of a multiple-screen displaysystem 100, which has a controller 105 (not shown), a monitoring module110 (not shown), and an adjustment module 120 (not shown) as shown inFIG. 1. The multiple-screen display system 100 may comprise a computer,such as a notebook PC 210, and a secondary monitor 220. The notebook PC210 may include a main monitor 211, the controller 105 and an imagecapturing device, such as a built-in camera 212 on the main monitor 211.The secondary monitor 220 may include a screen 221 and a built-in camera222. The secondary monitor 220 may be connected to the notebook PC 210via a cable 225 to the main monitor 211.

Referring to FIG. 2B, in an exemplary method of setting multiple screenlogical positions when multiple screens are connected via a cable, astep 230 may involve a secondary monitor (e.g., secondary monitor 220)being connected to a notebook PC (e.g., notebook PC 210). In a step 240,the secondary monitor (also known as a plug-and-play device, forexample), may send a signal to the notebook PC. In a step 250, theconnection of the plug-and-play device to the notebook PC may bedetected by the operating system of the notebook PC. In a step 260, acontroller (e.g., controller 105) may send a command to a monitoringmodule (e.g., monitoring module 110), which in turn may send an imagecapturing command to the secondary monitor and the main monitor.

In a step 270, a camera (e.g., camera 222) on the secondary monitor maycapture an object image, such as a face image of a user. In a step 290,the camera on the main monitor may capture the same object image at thesame time. The monitoring module may receive the images captured bothfrom the camera of the secondary monitor and the camera of the mainmonitor. The images from the camera of the secondary monitor maycomprise the position, size, and the angle of the face image of theuser. The image from the camera of the main monitor may include theposition, size, and the angle of the face image of the user.

In a step 282, the monitoring module may analyze the images of the userand angles of the cameras on the screen displays. The analysis mayinclude running face recognition process. The face recognition processmay include steps of extracting the face image of the user from eachimage captured from each camera of each monitor and analyzing thebrightness and direction of the face. More specifically, the facerecognition process may detect an eye position in the vertical directionfrom the image frame and detect a nose position in the horizontaldirection based on the detected eye position. The face recognitionprocess may compare detecting results with a table and determine theface orientation.

By comparing the images, monitoring module may analyze the relativeposition of each monitor. In a step 284, if the monitoring moduledetermines the locations and sizes of a face of the user in analyzingthe face image of the user, the monitoring module may determine thedisplays' physical layout, such as the secondary monitor situatingeither at the left of the main monitor or at the right of the mainmonitor in front of a user, for example. The displays' physical layoutmay be determined by comparing the orientation of the face imagesobtained from the cameras on the main monitor and the secondary monitor.When orientation of the face image obtained from the camera on thesecondary monitor is oriented to the left more than the orientation offace image from camera on the main monitor, the monitoring module maydetermine that the secondary monitor is situated on the left hand sidein physical layout relative to the main monitor in front of the user.When the orientation of face image from the camera on the secondarymonitor is oriented to the right more than that from the main monitor,the monitoring module may determine that the secondary monitor issituated on the right hand side relative to the main monitor in front ofthe user. If the monitoring module fails to analyze the face image ofthe user or fails to determine the locations and sizes of the face ofthe user due to various reasons, such as a dark background or not enoughimages captured by the camera, in the step 260, the monitoring modulemay send another image capturing command to the main monitor and thesecondary monitor until the monitoring module has succeeded indetermining the locations and sizes of a face of the user in the facerecognition process. Then in a step 286, an adjustment module (e.g.,adjustment module 120) may adjust the monitor settings of the notebookPC such that the logical positions of the monitors match the physicalarrangement of the monitors.

FIG. 3A refers to another exemplary embodiment of a multiple-screendisplay system 100, which may comprise a controller 105 (not shown), amonitoring module 110 (not shown), and an adjustment module 120 (notshown), as shown in FIG. 1. The multiple-screen display system 100 maycomprise a notebook PC 310, and a secondary monitor 320, such as atablet PC or a smart phone with a smart display (e.g., iPad®), forexample. The notebook PC 310 may include an embedded camera 312 and thecontroller 105. The secondary monitor 320 may have an embedded camera322 and a monitoring module 110.

Referring to FIG. 3B, a secondary monitor (e.g., secondary monitor 320)may be connected to a notebook PC (e.g., notebook PC 310) wirelessly. Ina step 330, the secondary monitor may launch a sub-display application.In a step 340, the secondary monitor may request the notebook PC forconnection with the secondary monitor. In a step 350, a controller(e.g., the controller 105) in the notebook PC may detect the secondarymonitor (or sub-display). In a step 360, the controller may send animage capturing command to the secondary monitor and the notebook PC.

In a step 370, a camera (e.g., camera 322) on the secondary monitor maycapture an object image, such as a face image of a user. In a step 384,the camera on the notebook may capture the same face image of the userat the same time. The image from the camera of the secondary monitor mayinclude the position, size, and the angle of the face image of the user.

In a step 380, a monitoring module (e.g., monitoring module 110) in thesecondary monitor may analyze the image data from the camera and runface recognition. If the face recognition fails to determine thelocations and sizes of the face of the user due to various reasons, suchas a dark background or not enough images captured by the camera, in thestep 370, the monitoring module may send a command to recapture images.Images may be recaptured by the camera. If the face recognition succeedsin determining the locations and sizes of the face of the user, themonitoring module may send face recognition results to the notebook PCfor comparison with results from face recognition from the PC.

In a step 386, the monitoring module in the notebook PC may analyze theimage data from the camera and run the face recognition program. If theface recognition fails to determine the locations and sizes of the faceof the user due to various reasons, such as a dark background or notenough images captured by the camera, in the step 384, the monitoringmodule may send a command to recapture the images of a user's face bythe camera in the notebook PC. If the face recognition succeeds indetermining the locations and sizes of the face of the user, in a step387, the monitoring module may compare face intensity of the user fromeach monitor.

By comparing the face intensity of the user, monitoring module of thenotebook PC may determine the displays' physical layout, such as thesecondary monitor situating either at the left of the notebook PC or atthe right of the notebook PC, for example. In a step 389, an adjustmentmodule (e.g., adjustment module 120) in the notebook PC may adjust themonitor settings of the notebook PC such that the logical positions ofthe notebook PC and the monitor match the physical arrangement of thenotebook PC and monitor.

Alternatively, if there is no monitoring module existing in thesecondary monitor, that is, if smart displays do not have a facerecognition program, in the step 386, images captured by camera in thesecondary monitor may be sent to the notebook PC without going throughthe step 380 and step 387. The process may proceed as described in FIG.2B.

It should be understood, of course, that the foregoing relate toexemplary embodiments of the invention and that modifications may bemade without departing from the spirit and scope of the invention as setforth in the following claims.

We claim:
 1. A computer system comprising: a plurality of displaydevices having at least one main display device and one secondarydisplay device; an image capturing device carried by the plurality ofdisplay devices; a controller coupled to at least one display device andthe image capturing devices; and an adjustment module adapted to setlogical positions of a plurality of display devices
 2. The computersystem of claim 1, wherein the secondary display device is adapted toconnect to a main display device via a cable.
 3. The computer system ofclaim 1, wherein the secondary display device is adapted to connect to amain display device wirelessly.
 4. The computer system of claim 1,wherein the image capturing devices carried by each of the plurality ofdisplay devices capture an object image at the same time.
 5. Thecomputer system of claim 4, wherein the object image is a face of auser.
 6. The computer system of claim 1, wherein the controller furthercomprises a monitoring module adapted to capture and analyze an objectimage.
 7. The computer system of claim 1, wherein the secondary displaydevice comprises a smart phone, or a tablet computer.
 8. A method ofsetting a multiple screen computer system comprising a plurality ofdisplay devices operatively associated with at least one computingdevice, the method comprising: capturing an object image; analyzing theobject image; and setting logical positions of the plurality of displaydevices based on the analysis of the object image.
 9. The method ofclaim 8 further comprising comparing the object images captured by animage capturing device on each of the plurality of display devices. 10.The method of claim 8 further comprising sending a request to an imagecapturing device of each display device to capture the object image. 11.The method of claim 8 further comprising extracting the object imagefrom each captured image.
 12. The method of claim 8 further comprisingdetermining a relative position of the plurality of display devices. 13.The method of claim 8 further comprising launching a sub-displayapplication.
 14. A computer readable medium having computer usableprogram code embodied therewith, the computer program code comprising:computer program code configured to launch a sub-display applicationexecuting a face recognition program; and computer program codeconfigured to set logical positions of a plurality of display devicesaccording to face images received by the face recognition program. 15.The computer program code of claim 14 further comprising computerprogram code configured to send a request to an image capturing deviceof each display device to capture the face images.
 16. The computerprogram code of claim 14 further comprising computer program codeconfigured to capture the face images of a user by using an imagecapturing device.
 17. The computer program code of claim 14 furthercomprising computer program code configured to extract the face imagesof a user.
 18. The computer program code of claim 14 further comprisingcomputer program code configured to analyze the face images of a user.19. The computer program code of claim 14 further comprising computerprogram code configured to compare the face images of a user from animage capturing device on each of a plurality of the display devices.20. The computer program code of claim 14 further comprising computerprogram code configured to determine a relative position of theplurality of display devices.